We first examine how H -consistency bounds vary across surrogates based on the number of classes.

Moreover,the proposed multi-class kernel learning algorithms have statistical guarantees and fast convergence rates. Experimental results on lots of benchmark datasets show that our proposed methods can significantly outperform the existing multi-class classification methods. The major contributions ofthispaper include: 1)Anewlocal Rademacher complexitybased bound withfastconvergence rate for multi-class classification is established. Existing works [16,27] for multi-class classifiers with Rademacher complexity does not take into account couplings among different classes.

Learning objective is a fundamental component in any machinelearningsystem.

A.10 Extensiontomulti-class As explained at the beginning, our algorithm can largely extend to the multi-class/group setting. The task is to predict whether an individual's income exceeds50K.Thedatasetconsists of48,842examples and28features. Fairface, the face attribute dataset containing 108,501 images with balanced race and gender groups [15]. We use a pre-trained vision transformer (ViT/B-32) model [8] to extract image representations, and project them into 50-dimensional feature vectors. For constrained learning, we categorize race into White and Non-White groups.

Active learning aims to reduce the number of labeled examples needed to train deep networks, but the empirical performance of active learning algorithms can vary dramatically across datasets and applications.

These are upper bounds on the zero-one estimation error of any predictor in a hypothesis set, expressed in terms of its surrogate loss estimation error.

Classification is a fundamental task in science and engineering on which machine learning methods have shown outstanding performances. However, it is challenging to determine whether such methods have achieved the Bayes error rate, that is, the lowest error rate attained by any classifier. This is mainly due to the fact that the Bayes error rate is not known in general and hence, effectively estimating it is paramount. Inspired by the work by Ishida et al. (2023), we propose an estimator for the Bayes error rate of supervised multi-class classification problems. We analyze several theoretical aspects of such estimator, including its consistency, unbiasedness, convergence rate, variance, and robustness. We also propose a denoising method that reduces the noise that potentially corrupts the data labels, and we improve the robustness of the proposed estimator to outliers by incorporating the median-of-means estimator. Our analysis demonstrates the consistency, asymptotic unbiasedness, convergence rate, and robustness of the proposed estimators.

This paper presents a comprehensive analysis of the growth rate of $H$-consistency bounds (and excess error bounds) for various surrogate losses used in classification. We prove a square-root growth rate near zero for smooth margin-based surrogate losses in binary classification, providing both upper and lower bounds under mild assumptions.